Robustness testing of a distributed simulation backplane

◆ This research is built upon and extends the Ballista project. • High level testing done using the API to perform fault injection – Send exceptional values into a system through the API – Requires no modification to code -only linkable object files needed • Each test is a specific function call with a specific set of parameters – Combinations of valid and invalid parameters tried in turn ◆ Yes, Ballista can be extended and it turned out to be easy! ◆ Applied Ballista to a general-purpose distributed system software used for military simulations • Specifically engineered for robustness • They weren’t perfect -but usually they weren’t too bad either • Ballista found the weak spots that they should concentrate on (a “profiling” tool for robustness!) ◆ Porting exception handling code seems to be a problem Overview: Testing the RTI A Ballista is an ancient siege weapon for hurling objects at fortified defenses. ◆ System Robustness • Motivation • Ballista Automatic Robustness Testing Tool ◆ Enhancements for RTI Testing • Results of Ballista testing on 4 versions of HLA RTI (86 functions) • Data analysis of results (77,000 tests) • Comparison to Operating Systems • Segmentation faults vs. RTI Internal Error exception ◆ Issues and Future Direction • Extending Ballista to other application areas • Creating a general-purpose, scalable testing framework ◆ Current Application: High-Level Architecture Run-Time Infrastructure (HLA RTI)